TCCT: Tightly-coupled convolutional transformer on time series forecasting

Time series forecasting is essential for a wide range of real-world applications. Recent studies have shown the superiority Transformer in dealing with such problems, especially long sequence time input(LSTI) and forecasting(LSTF) problems. To improve efficiency enhance locality Transformer, these combine CNN varying degrees. However, their combinations are loosely-coupled do not make full use CNN. address this issue, we propose concept tightly-coupled convolutional Transformer(TCCT) three TCCT architectures which apply transformed into Transformer: (1) CSPAttention: through fusing CSPNet self-attention mechanism, computation cost mechanism reduced by 30% memory usage 50% while achieving equivalent or beyond prediction accuracy. (2) Dilated causal convolution: method to modify distilling operation proposed Informer replacing canonical layers dilated gain exponentially receptive field growth. (3) Passthrough mechanism: application passthrough stack blocks helps Transformer-like models get more fine-grained information negligible extra costs. Our experiments on datasets show that our could greatly performance existing state-of-art much lower costs, including LogTrans Informer.